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DETAILED ACTION 



Response to Amendment 

1. The amendments, filed on 08/08/2008, have been entered and made of record. Claims 1-8, 
10-36, 41-42, and 44-51 are pending. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1-8, 10-36, 41-42, and 44-51 have been fully 
considered but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §102 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

4. Claims 1-3,5,8, 17-18, 21, 23-25, 30-31, 34, 36, and 44-51 are rejected under 35 
U.S.C. § 102(e) as being anticipated by Yonezawa (US 7,187,402). 

Regarding claim 1, Yonezawa discloses a method of presenting a user with a multimedia 
experience (graphical user interface GUI of the monitoring system) corresponding to a venue 
(the arrangement of seats in an office or store), the method comprising: 

- managing a sensor array having at least two sensors (camera selecting buttons for 
a plurality of cameras, Figs 32-33) that are configured to provide a stream of data 



Application/Control Number: 10/748,123 Page 3 

Art Unit: 2622 

units (Fig 3 shows a stream of data units, camera icons 521, 522, 523, . . .) (7:15- 
41 and 16:30-39); 

determining locations for the sensors in the sensor array to provide location 
information (the camera managing server 50 holds static information related to 
the respective cameras, such as the locations at which the respective cameras are 
installed) (17:27-46); 

associating the location information with the sensors in the sensor array (the 
camera control server 52 displays the locations and directions where both the 
cameras connected to the monitoring terminal 40 and all the available cameras 
through the network 46 are installed so that predetermined corresponding camera 
symbols overlap with the map on the display screen of the monitoring terminal 
40) (17:47-57); 

enabling the user to perceive a map (a map window 60, Fig 36) related to the 
venue (a map that shows the arrangement of seats in an office or store is displayed 
on the map window 60) (17:64-18:27); 

- relating the perceived map (a map window 60, Fig 36) to one or more of the 
sensors in the sensor array (on the map window 60, the camera icons 66-1 to 66- 
12 which correspond to the respective locations of cameras installed in such an 
office or store are displayed) (18:13-27); 

- receiving a request from the user identifying a selected position within the map 
(permits a user to freely rearrange the display positions of video camera images so 
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as to be easy to handle, and to select video camera images to be displayed 
simultaneously) (1 1 :21-35); 

identifying one or more of the sensors in the sensor array corresponding to the 
selected position (permits a user to identify and to select video camera images to 
be displayed simultaneously, thereby giving flexibility to operation of the 
monitoring system) (1 1 :21-35); and 
- presenting to the user the multimedia experience (graphical user interface GUI of 
the monitoring system) based on one or more streams of data units associated 
with the identified sensors (a state in which the map 520 among the maps 510 to 
540 is displayed on the map display area 502 and in which camera icons 521, 522, 
523, . . . located on the map 520 are displayed as in Fig 3 which with clicking the 
tag of map 530, the map 530 is displayed on the map display area 502 as shown in 
Fig 4, and at the same time, camera icons 531, 532, 533, . . . located on the map 
530 are displayed) (6:1-23). 
Regarding claim 2, Yonezawa discloses more than one sensor in the sensor array is 
identified (camera selecting buttons for a plurality of cameras, Figs 32-33), and wherein 
presenting to the user the multimedia experience includes providing a multimedia experience 
based on streams of data received from the more than one identified sensors (permits a user 
to freely rearrange the display positions of video camera images so as to be easy to handle, 
and to select video camera images to be displayed based on a stream of data units, camera 
icons 521,522, 523, . . .) (7:15-41, 11:21-35 and 16:30-39). 
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Regarding claim 3, Yonezawa discloses managing the sensor array (the camera managing 
server 50, Fig 35) includes operating multiple camera systems (a plurality of cameras, Figs 
32-33), the camera systems each including a video capture system and a location provider 
system (the camera managing server 50 holds static information related to the respective 
cameras, such as the locations at which the respective cameras are installed, and dynamic 
information representing the present condition as to whether or not the cameras are being 
controlled or images from the cameras are being displayed, and the names of the terminals 
provided with the function of video composing as the map window 60 displays camera icons 
which represent the locations) (17:27-46). 

Regarding claim 5, Yonezawa discloses operating the multiple camera systems (a 
plurality of cameras as indicating in the camera selecting buttons, Figs 32-33) includes 
operating two or more camera systems that provide video (i.e. camera icons 521, 522, 523, . 
...Fig 3) (7:15-41 and 16:30-39). 

Regarding claim 8, Yonezawa discloses managing the sensor array (a plurality of 
cameras) include managing more than one type of sensor (various sensors for different static 
information related to the respective cameras, such as the names of the respective cameras, 
the names of the terminals to which the cameras are connected, the locations at which the 
respective cameras are installed, whether or not the panning, tilting and zooming of the 
cameras can be controlled, and dynamic information representing the present condition as to 
whether or not the cameras are being controlled or images from the cameras are being 
displayed, and the names of the terminals provided with the function of video composing) 
(17:27-46). 
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Regarding claim 17, this claim recites same limitations as claim 1. Thus it is analyzed 
and rejected as previously discussed with respect to claim 1 above. 

Regarding claim 18, Yonezawa discloses notifying the user about the availability of the 
better-matching sensor (video camera) includes enabling the user to receive to the stream of 
data units (maps 510-540, Fig 3) from the better matching sensor (a computer for controlling 
the video camera 10 connected to the camera control circuit 12 by sending a control code to 
the camera control circuit 12 and for transmitting video data obtained from the video camera 
10 through a network interface 38 to the network) (4:24-34). 

Regarding claim 21, Yonezawa discloses presenting to the user the multimedia 
experience (graphical user interface GUI of the monitoring system) includes combining the 
one or more stream of data units (maps 5 10-540, Fig 3) with other streams of data units from 
other sensors in the sensor array (camera selecting buttons for a plurality of cameras, Figs 32- 
33) into a combined stream of data units and enabling the user to access the combined stream 
of data units (6:1-23). 

Regarding claim 23, Yonezawa discloses combining the one or more stream of data units 
(maps 510-540, Fig 3) includes enabling presentation of a simulated view from a location 
where no sensor is located (Fig 5 shows only the six video display areas but no sensor icon is 
present) (6:24-41 and Fig 5). 

Regarding claim 24, Yonezawa discloses presenting to the user the multimedia 
experience (graphical user interface GUI of the monitoring system) includes performing 
intermediary processing on the one or more streams of data units (maps 5 10-540, Fig 3) to 
generate an edited stream of data units and enabling the user to access the edited stream (a 
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state in which the map 520 among the maps 510 to 540 is displayed on the map display area 
502 and in which camera icons 521, 522, 523, . . . located on the map 520 are displayed as in 
Fig 3 which with clicking the tag of map 530, the map 530 is displayed on the map display 
area 502 as shown in Fig 4, and at the same time, camera icons 531, 532, 533, . . . located on 
the map 530 are displayed) (6:1-23). 

As to claim 25, this claim differs from claim 1 only in that the claim 1 is a method claim 
whereas claim 25 is apparatus. Thus the apparatus claim 25 is analyzed and rejected as 
previously discussed with respected to claim 1 above. 

Regarding claims 30 and 31, these claims recite same limitations as claims 17 and 18, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 17 and 18 above. 

Regarding claims 34 and 36, these claims recite same limitations as claims 21 and 23, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 21 and 23 above. 

Regarding claim 44, Yonezawa discloses a method of presenting a user with a 
multimedia experience (graphical user interface GUI of the monitoring system) 
corresponding to a venue (the arrangement of seats in an office or store), the method 
comprising: 

- managing a first sensor (i.e. camera 521) and a second sensor (i.e. camera 523) 
(Fig 6 and 6:1-23); 

determining, at a first time (i.e. map 5 1 0, Fig 6), a first location of the first sensor 
(521) and a first location of the second sensor (523) (Fig 6 and 6:1-23); 
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enabling display (a map window 500, Fig 6), to a user, of a map related to the 
venue (a layout of an office, a shop, or a warehouse) (Fig 6 and 6:1-23); 
enabling display (a map window 500, Fig 6), to the user, of the first sensor's first 
location and the second sensor's first location (map 510, Fig 6) on the map related 
to the venue (a layout of an office, a shop, or a warehouse) (Fig 6 and 6:1-23); 

- determining, at a second time (i.e. map 520, Fig 6), a second location of the first 
sensor (521) and a second location of the second sensor (523), wherein the second 
time is after the first time (the user can select one specified camera for displaying 
an image by selecting one button from these buttons on the screen which permits 
a user to freely rearrange the display positions of video camera images so as to be 
easy to handle, and to select map 510 before map 520) (1 1 :2 1-35 and 15:61- 
16:11), 

- the first sensor's second location is different from the first sensor's first location, 
and the second sensor's second location is different from the second sensor's first 
location (Fig 3 shows a state in which the map 520 among the maps 510 to 540 is 
displayed on the map display area 502 and in which camera icons 521, 522, 523, . 
. . located on the map 520 are displayed as a plurality of maps 520, 530, 540, . . . ; 
e.g. map 510 represents a first location of a office, map 520 represents a second 
location of a shop or a warehouse, as each indicating a different layout of an 
office, a shop, or a warehouse) (6:1-23); 
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enabling display (a map window 500, Fig 6), to the user, of the first sensor's (521) 
second location and the second sensor's (523) second location on the map (map 
520) related to the venue (the arrangement of seats in an office or store) 6:42-7:9); 

- receiving a request, from the user, identifying one of the first sensor and the 
second sensor (permits a user to freely rearrange the display positions of video 
camera images so as to be easy to handle, and to select video camera images to be 
displayed simultaneously) (1 1:21-35); and 

- enabling display (a map window 500, Fig 6), to the user, of the multimedia 
experience (graphical user interface GUI of the monitoring system) based on one 
or more streams of data received from the identified one of the first sensor and the 
second sensor (a state in which the map 520 among the maps 510 to 540 is 
displayed on the map display area 502 and in which camera icons 521, 522, 523, . 

. . located on the map 520 are displayed as in Fig 3 which with clicking the tag of 
map 530, the map 530 is displayed on the map display area 502 as shown in Fig 4, 
and at the same time, camera icons 531, 532, 533, . . . located on the map 530 are 
displayed) (6:1-23). 

Regarding claim 45, Yonezawa discloses the first sensor's second location is different 
from the first sensor's first location, and the second sensor's second location is different from 
the second sensor's first location (Fig 3 shows a state in which the map 520 among the maps 
510 to 540 is displayed on the map display area 502 and in which camera icons 521, 522, 
523, . . . located on the map 520 are displayed as a plurality of maps 520, 530, 540, . . . ; e.g. 
map 510 represents a first location of a office, map 520 represents a second location of a 
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shop or a warehouse, as each indicating a different layout of an office, a shop, or a 
warehouse) (6:1-23); 

Regarding claim 46, Yonezawa discloses enabling display (a map window 500, Fig 6), to 
the user, of the first sensor's second location and the second sensor's second location (map 
520, Fig 6) on the map related to the venue (a layout of an office, a shop, or a warehouse) 
(Fig 6 and 6:1-23) includes removing, from the map, the first sensor's first location and the 
second sensor's first location (maps 520, 530, 540, . . . , each indicating a different layout of 
an office, a shop, or a warehouse, thus each map has a different layout of the sensor's 
locations, e.g. an office, a shop, or a warehouse) (6:1-23). 

Regarding claim 47, Yonezawa discloses the sensor array includes managing a camera 
system (the camera managing server 50 is software to manage all the cameras, connected to 
the network 46, which can be utilized by the monitoring terminal 40) (7:27-46), the camera 
system including a video sensor and an audio sensor (a camera input selector 14 selects 
which of the video cameras 10-1 to 10-3 to control, in order to take in an output signal from 
the selected camera, and the output signal is normally a video signal, but both a video signal 
and an audio signal when a microphone-equipped camera is used) (14:60-15:3). 

Regarding claim 48, Yonezawa discloses enabling the user to perceive the map includes 
enabling display (a map window 60, Fig 36), to the user, of the relative orientation and scale 
of physical elements of the entertainment event or venue, other than the streams of data units 
associated with the sensors in the sensor array (a map that shows the arrangement of seats in 
an office or store is displayed on the map window 60, and on this map the camera icons 66-1 
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to 66-12 which correspond to the respective locations of cameras installed in such an office 
or store are displayed) (17:64-18:27). 

Regarding claim 49, this claim recites same limitations as claim 48. Thus it is analyzed 
and rejected as previously discussed with respect to claim 48 above. 

Regarding claim 50, Yonezawa discloses identifying the one or more of the sensors in the 
sensor array (Fig 3 shows a stream of data units, camera icons 521, 522, 523, . . .) 
corresponding to the selected position includes: identifying multiple sensors in the sensor 
array (the camera control server 52 displays the locations and directions where both the 
cameras connected to the monitoring terminal 40 and all the available cameras through the 
network 46 are installed so that predetermined corresponding camera symbols overlap with 
the map on the display screen of the monitoring terminal 40) (17:47-57); and identifying two 
or more sensors in the sensory array in proximity to the selected position, the two or more 
sensors in the sensory array being fewer than all of the multiple sensors in the sensor array 
(permits a user to identify and to select video camera images to be displayed simultaneously, 
thereby giving flexibility to operation of the monitoring system, and Fig 3 shows a stream of 
data units, camera icons 521, 522, 523, . . .) (7:15-41, 11:21-35 and 16:30-39). 

Regarding claim 51, Yonezawa discloses presenting to the user the multimedia 
experience (graphical user interface GUI of the monitoring system) based on the one or more 
streams of data units associated with the identified sensors includes combining the streams of 
data units associated with the two or more sensors in closest proximity to the selected 
position (a state in which the map 520 among the maps 5 10 to 540 is displayed on the map 
display area 502 and in which camera icons 521, 522, 523, . . . located on the map 520 are 
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displayed as in Fig 3 which with clicking the tag of map 530, the map 530 is displayed on the 
map display area 502 as shown in Fig 4, and at the same time, camera icons 531, 532, 533, . . 
. located on the map 530 are displayed) (6:1-23). 

Claim Rejections - 35 USC §103 

5. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

6. Claims 4, 10-12, and 41-42 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Yonezawa (US 7,187,402) in view of Valleriano (US 2005/0093976). 

Regarding claim 4, the limitations of claim 1 are taught above, Valleriano does teach 
determining locations for the sensors in the sensor array (digital video cameras 20, Fig 1) 
includes using the location provider system of each of the camera systems to determine 
location information (position information) using a Global Positioning system receiver (a 
GPS technologies) ([0056], Valleriano). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to choose the GPS technologies as taught by Valleriano into 
McClintock's video viewing and recording system, so as to easily determine where a camera 
is located and the direction of its line of sight ([0056], Valleriano). 

Regarding claim 10, the limitations of claim 1 are taught above, Valleriano does teach 
determining the location for the sensor (camera position) includes determining the location 
(three-dimensional location data) relative to an architectural structure (tracking system 80) 
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for the sensor in an entertainment venue (a sports event such as a soccer game) ([0057], 
Valleriano). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to choose the tracking system as taught by Valleriano into McClintock's 
video viewing and recording system, so as to easily correlate asynchronously captured event 
data and images in a three dimensional volumetric space on a playing field ([0014]-[0015], 
Valleriano) 

Regarding claim 1 1, the limitations of claim 1 are taught above, Valleriano does teach 
using the location for the sensor (camera position) in the entertainment venue (a sports event 
such as a soccer game) to determine metadata descriptive (i.e. camera ID, photographer ID, 
camera line-of-sight data and a field of view, Fig 1) of the entertainment experience ([0055], 
[0057], Valleriano). 

Regarding claim 12, the limitations of claim 1 are taught above, Valleriano does teach 
enabling the user to perceive and relating the perceived map includes using metadata (i.e. 
camera ID, photographer ID, camera line-of-sight data and a field of view, Fig 1) to describe 
the user experience associated with the sensor ([0055], [0057], Valleriano). 

Regarding claims 41 and 42, these claims recite same limitations as claim 4. Thus they 
are analyzed and rejected as previously discussed with respect to claim 4 above. 
7. Claims 14-16, 19-20, 27-29 and 32-33 are rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Yonezawa (US 7,187,402) in view of McClintock (US 5,598,208). 

Regarding claim 14, the limitations of claim 1 are taught above, Yonezawa does not 
disclose determining a permission level for the user. However, McClintock discloses 
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determining a permission level for the user (the remote control 116 which regulates access to 
each compartment as each user would pay in order to gain access to the system) (8:18-31, 
McClintock). 

Thus, it would have been obvious to one of ordinary skill in the art to have included the 
access control as taught by McClintock into Yonezawa's communication apparatus, as to 
provide a as such a control of programming means via wireless transmission with sensors 
which are coupled to a video system at various locations throughout the event venue (8:18- 
31, McClintock). 

Regarding claim 15, the limitations of claims land 14 are taught above, Yonezawa does 
not disclose determining the permission level includes determining a level of access to which 
the user has subscribed. However, McClintock discloses determining the permission level 
includes determining a level of access to which the user has subscribed (each user could be 
supplied with a code index which informs the user of a unique identification code to join in 
the action) (7:47-66, McClintock). 

Regarding claim 16, the limitations of claims 1 and 14 are taught above, Yonezawa does 
not disclose determining the permission level includes identifying sensors. However, 
McClintock discloses determining the permission level (regulates access) includes 
identifying sensors (coupled to a video system at various locations throughout the event 
venue) that are accessible and inaccessible to the user, and regulating access (regulates 
access) by the user in response to the permission level (user would key in with the remote 
control 116, Fig 6B) (8:18-31, McClintock). 
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Regarding claim 19, the limitations of claims 1 and 17 are taught above, Yonezawa does 
not disclose notifying the user about the availability of the better-matching sensor. However, 
McClintock discloses notifying the user about the availability of the better-matching sensor 
(video camera) includes enabling the user to upgrade a permission level so that the user may 
receive a stream of data units from the better matching sensor (each user would pay in order 
to gain access to the system)(7:47-66, McClintock). 

Regarding claim 20, the limitations of claims 1 and 14 are taught above, Yonezawa does 
not disclose determining that the permission level supports access to the stream of data units 
from the better-matching sensor before enabling access to the stream of data units from the 
better matching sensor. However, McClintock discloses determining that the permission level 
supports access (regulates access) to the stream of data units from the better-matching sensor 
before enabling access to the stream of data units from the better matching sensor (receive an 
overair broadcast video signal from each of cameras) (9:54-10:5 and 10:28-44, McClintock). 

Regarding claims 27, 28, 29, and 33, these claims recite same limitations as claims 14, 
15, 16, and 20, respectively. Thus they are analyzed and rejected as previously discussed 
with respect to claims 14, 15, 16, and 20 above. 

Regarding claim 32, this claim recites same limitations as claim 19. Thus it is analyzed 
and rejected as previously discussed with respect to claim 19 above. 
8. Claims 13 and 26 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Yonezawa (US 7,187,402) in view of Bernardo (US 2002/0047895). 

Regarding claim 13, the limitations of claim 1 are taught above, Yonezawa does not 
disclose enabling the user to perceive the map and relating the perceived map. However, 
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Bernardo discloses enabling the user to perceive the map and relating the perceived map 
(retrieved composite image and map) includes generating a web page (a particular web page 
Fig 16) enabling the user to navigate among the sensors (video camera) in the sensor array 
(digital video cameras 10) and to select one or more of the sensors in the sensor array 
([0083]-[0086], Bernardo). 

Thus, it would have been obvious to one of ordinary skill in the art to have included the 
web page as taught by Bernardo into Yonezawa's video viewing and recording system, as to 
provide a hyperlink for retrieving and displaying the composite images and association 
information preferably on a separate browser window ([0086], Bernardo). 

Regarding claim 26, this claim recites same limitations as claim 13. Thus it is analyzed 
and rejected as previously discussed with respect to claim 13 above. 
9. Claims 6, 22 and 35 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Yonezawa (US 7,187,402) in view of Ritchey (US 5,495,576). 

Regarding claim 6, the limitations of claim 1 are taught above, Yonezawa does not 
explicitly disclose managing the sensor array and associating the location information 
includes operating multiple microphone systems, where the microphone systems include a 
sound capture system and a location provider system. However Ritchey does teach managing 
the sensor array (sensor array 36, Fig 5) and associating the location information (sensors 
location) includes operating multiple microphone systems (microphones 39a-39f, Figs 4-5), 
where the microphone systems (microphones 39a-39f) include a sound capture system 
(acoustical system) and a location provider system (10:17-30 and 13:24-60, Ritchey). 
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Thus, it would have been obvious to one of ordinary skill in the art to have included the 
acoustical system as taught by Ritchey into Yonezawa's video viewing and recording system, 
as to provide a multimedia system which could performing a spherical acoustical field of 
regard coverage about a location (13:25-61, Ritchey). 

Regarding claim 22, Ritchey discloses combining the one or more streams of data units 
includes presenting a three dimensional presentation (three-dimensional computer generated 
model that comprises the virtual reality system presented to a participant) (7:30-54, Ritchey). 

Thus, it would have been obvious to one of ordinary skill in the art to have included the 
three dimensional presentation as taught by Ritchey into McClintock and Valleriano's video 
viewing and recording system, as to provide a virtual reality/telepresence panoramic three 
dimensional images associated a three dimensional audio systems (7:30-54, Ritchey). 

Regarding claim 35, this claim recites same limitations as claim 22. Thus it is analyzed 
and rejected as previously discussed with respect to claim 22 above. 
10. Claim 7 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Yonezawa (US 
7,187,402) in view of Ritchey, and further in view of Bernardo (US 2002/0047895). 

Regarding claim 7, the limitations of claims 1 and 6 are taught above, Yonezawa and 
Ritchey do not explicitly disclose operating the multiple camera systems includes using the 
location provider system of the microphone system to determine location information using 
at least one of a Global Positioning system receiver, a gyroscope, and a local beacon. 
However Bernardo does teach operating the multiple camera systems (digital video cameras 
10, Fig 1) includes determining location information (position information) using at least one 
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of a Global Positioning system receiver (a GPS receiver 16, Fig 1), a gyroscope, and a local 
beacon ([0033]-[0034], Bernardo). 

Thus, it would have been obvious to one of ordinary skill in the art to have included the 
GPS receiver as taught by Bernardo into Yonezawa and Ritchey's video viewing and 
recording system, as to provide a more accurate calculation of the position information 
([0034], Bernardo). 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: Kato et al. (US 6,697,105), Suzuki et al. (US6,002,995), Kawai et al. (US 
6,680,746), Morota et al. (US 6,919,921), Oya et al. (US 6,208,379), Jain et al. (US 
5,745,126), Moezzi et al. (US 5,850,352), and Mottur et al. (US 7,199,817). 

Inquiries 

12. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

13. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Sinh Tran can be reached on 571-272-7564. The fax phone number for the organization 
where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status information 
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for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 
/Tuan V Ho/ 

Primary Examiner, Art Unit 2622 
KW 

29 October 2008 



